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ABSTRACT 

Learning English Electronically (LEE) , a computer 
software package designed for adult English as a Second Language 
(ESL) students enrolled in intermediate level community college ESL 
classes, was evaluated at Glendale Community College in Glendale, 
Arizona to assess student and teacher attitudes toward the program. 
LEE consists of 43 lessons emphasizing grammar concepts and accurate 
sentence structure, and covering topics such as employment, food, 
health, school, and transportation. Evaluation data were collected 
through student and teacher surveys, interviews, student test scores, 
and evaluator observations. Both students and teachers expressed very 
positive attitudes about LEE. Strengths of the program identified by 
teachers were the additional grammar practice available, the 
self-paced and non-threatening nature of the program, the inclusion 
of sound in the program, and the grammar topics being presented with 
content topics. Students enjoyed using the program, felt it helped 
them learn, and liked having teacher supervision while using the 
program individually. Students and teachers alike found that a 
weakness of the program was the slow response time of the computer in 
executing commands. Additional weaknesses included difficultly using 
the mouse, starting and ending the program, and printing. 
Recommendations for improvement, survey responses from teachers and 
students, end of module test scores, and evaluator observations are 
included. (SWC) 
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This paper describes a formative evaluation of Learning English Electronically (LEE) 
conducted at Glendale Community College in Glendale, Arizona. A description of the program, 
as well as posttest, survey, and observation data will be provided during the proposed 
presentation. Implications of the compiled data will also be discussed. 



Description of Learning English Electronically 

Learning English Electronically (LEE) was developed with a grant from Apple 
approximately six and a half years ago. It is a computer software package designed for adult 
English as a Second Language (ESL) students enrolled in intermediate level community college 
ESL classes. It is intended to be used as a supplement to ESL classroom instruction. 

The program consists of 43 different lessons emphasizing various grammar concepts and 
accurate sentence structure. The 43 lessons are divided into 14 sections. The individual lessons 
take approximately 25 to 35 minutes to complete. Each section focuses on a different English 
grammar concept. Each of the grammar concepts has from one to five different individual lessons 
covering five different life content topics: employment, food, health, school, and transportation. 

At the present time, LEE is incomplete because all of the five topics are not included in every 
grammar concept. The program is designed for adult ESL community college students who are 
enrolled in intermediate ESL classes and have a moderate level of proficiency in the English 
language, but have difficulty in writing accurate and effective sentences. 

The courseware objectives vary from section to section, but remain the same for each 
lesson within a particular section. The various objectives deal with identifying grammar concepts 
correctly in context, using different grammar concepts correctly in sentences, listing grammar 
concepts according to category, and stating different forms of various grammar concepts. These 
are just a few general examples of the objectives contained in LEE Overall, there are 35objectives 
among the 14 sections. The program package consists of the an Instructor's Guide, Student 
Lesson Sheets, and 43 lesson disks with one main menu disk. (Hyper Card 2.0 application disks 
are also included in the package.) 



Evaluation Procedures 

The evaluation was designed to assess student and teacher attitudes and judgments about 
Learning English Electronically. Performance data in the form of test scores were gathered to 
assess the effectiveness of the program. Data were collected through (1) a survey administered to 
both students and teachers (2) personal interviews conducted with students and teachers (3) test 
scores submitted by students at the end of each module, and (4) observations. 

The surveys were completed anonymously by both the students and teachers. Responses 
to the choice items on the surveys were scored on a 1 (strongly agree) to 5 (strongly disagree) 
scale. For the open-ended items, similar responses to an item were categorized together and all 
responses are listed in the summary tables in this report. 
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Personal interviews with both students and teachers were done on site at the Learning 
Center at Glendale Community College. Due to the fact that the students have difficulty with the 
English language, some questions from the survey were also asked in the personal interviews. 

The four topics covered in the student interviews were the method of use, areas of like and 
dislike, problems using the program, and miscellaneous comments. The four topics covered in the 
teacher interviews were the method of course implementation, program strengths and weaknesses, 
improvement suggestions, and whether or not the topics cover all the components of grammar 
they would like their students to know. 

Student performance data were also collected on site at the time of the personal 
interviews. As each student finished a module, he or she was asked to complete and print out the 
test at the end and submit it to the evaluators. Scores are given as the number correct out of the 
total number of questions for Sections A, B, and C of the test, A being the easiest section and C 
being the most difficult. There are also two questions in which the student can construct original 
sentences and the teacher scores them after they are printed out. Each sentence is worth five 
points. This section will not be considered in the evaluation due to the difficulty in assigning point 
values consistently to the different components of the sentences. Parts A, B, and C will be scored 
on the percentage of questions answered correctly out of the total number of questions. 

Several other evaluation tasks were performed as part of the overall evaluation of the 
program. Reference manuals and several modules from the software itself were reviewed. Also, 
on site observational data were gathered as the evaluators were able to view the students actually 
using LEE and interacting with their instructors. 

Results 

The results are summarized below, first for the teachers, and then for the students. These 
are followed by performance test score data and observational data. 

Teachers Survey Responses 

Responses to the survey are summarized in Table 1 for the three teachers who completed 
the survey. The summary shows the number of participants selecting each response choice, the 
average score on each item on a scale of 1 (most favorable) to 5 (least favorable), and the overall 
mean score for the group. 

The overall mean rating by teachers for Learning English Electronically was 1.71, a very 
favorable rating indicating agreement to strong agreement with positive statements about the 
course. The highest rated items (1 .33) were item 2 (The skills taught in the modules are valuable 
for my students), item 4 (The stories aid in the learning process), and item 7 (There are enough 
practice exercises). The lowest rated items (2.00) were item 1 (The modules are well organized), 
item 5 (The rules are clearly stated), item 6 (Instructions for the exercises are clear), and item 8 
(The modules are about the right length). 

The teachers thought the strengths of the program were that there was additional grammar 
practice available for the students if they took advantage of it, that the program was self-paced 
and non-threatening, that it has sound, and that the grammar topics were presented with content 
topics. The major weakness of the program was cited as the slow response of the computer in 
executing commands, while the potential improvements included making ESL students more 
aware of the program and increasing the speed of the computer. All three teachers stated that 
they used the program as a supplement to their classes and two said that they would use it again. 
One teacher commented that the program needs to be completed so that the students have access 
to more modules. 
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Teacher Interviews 

The interviews with the two teachers dealt with method of course implementation, 
program strengths and weaknesses, and suggestions for improvements. The teachers were also 
asked if they felt that the 14 topics fully covered all the components of grammar that they wanted 
students to know. 

Both teachers said that they used the program strictly as a supplement to their normal 
course curricula and that students used it on their own time. One teacher stated that she would 
not feel comfortable requiring the students to use the program exclusively for class until the flaws 
in it were corrected. 

When asked about the strengths of the program, one teacher stated that doing a grammar 
lesson with several different topic choices (i.e. employment, food, etc.) while still learning the 
main focus of the module was interesting as well as effective for the students. Also, seeing 
grammar rules applied in the stories in context was good practice for the students. The other 
teacher stated that the program allowed students to work at their own pace and was a 
nonthreatening/nonembarrassing way for students to practice their English. 

Both teachers reported the fact that the students couldn't print out any screen of their 
choice, just end-of-module tests and some sentence writing exercises was a weakness of LEE. 
Other weaknesses cited were slowness in accessing information and some incorrect grammar and 
punctuation in the program. 

Both teachers reported that the 14 topics in the program fully covered all the components 
of grammar that their students needed to know. 

Recommendations for improvements in the program included increasing the speed in 
accessing information, improving the overall sound quality, and completing the content topics for 
all the modules. 

Student Survey Responses 

Table 2 shows the survey responses for the nine students who completed the survey. 

The mean overall rating of the program by the students was 1 .65, a very favorable rating. 
Item 10 (I liked using the program) and item 3 (the practice exercise helped me learn) received 
the most favorable ratings of 1.25 and 1.38 respectively. Item 1 (The program was easy to use) 
and item 4 (The instructions were easy to understand) received the least favorable ratings of 2.1 1 
for each item. One of the students thought the program was too short, while five thought it was 
about the right length. Seven of the students thought the modules were about right in ease or 
difficulty of use. Four students said that they used the modules on their own while four said that 
they used the modules because the teacher assigned them. 

Three students stated that the thing they liked least about the program was that it works 
too slowly, while one student stated that any screen they desired could not be printed. 

Recommendations for improvement included making the response time faster (2), 
including more exercises and tests (1), and providing more information about the program 
throughout the college (1). 

Student Interviews 

Personal interviews were held with nine students on site at Glendale Community College 
in the Learning Center. Topics covered included the method of use, best and least liked features, 
problems with use, and miscellaneous comments. 

All nine students said that they used LEE on their own, but with teacher supervision (in 
the Learning Center). Three of the students said that the exercises and instruction were what they 
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liked best about the program, while the slow response time of the computer was what they liked 
the least. Difficulty using the mouse as well as starting and ending the program were cited as 
problems with the program. General comments were favorable overall. Most students 
interviewed said they liked the program, learned a lot from it, and would use it again. 

Test Scores 

Module-end performance tests were completed by five students. Not every student 
completed every section, so only those sections with scores are reported in Table 3. The tests 
consist of sections A, B, and C, section A having the easiest items and section C having the most 
difficult. Test results are given based on the percentage of items correct out of the total number 
of items in a section. A mean score is given for each section. 

Observational Data 

Observational data were gathered at the Learning Center at Glendale Community College over a 
span of four hours one morning. The Learning Center has several sites for Learning English 
Electronically and nine students used the program during the observational time period. 

Evaluator observations are reported in Table 4. 

Discussion 

The dominant finding from this evaluation is that both teachers and students think that 
Learning English Electronically is a good program. Both groups expressed very positive attitudes 
toward it. 

Module-end performance test scores indicated that most students found that Section A 
was challenging for them, but that they could still obtain a moderate level of success completing 
it. (The mean score for this section was seventy-three percent.) While Sections B and C were not 
as frequently completed due to the level of difficulty, they were successfully accomplished by a 
few students. (The mean scores for these sections were both seventy-five percent, but one 
student did score a hundred percent on each section.) Regarding these data, one suggestion for 
improvement would be to perhaps make some easier test items for each level so that the students 
may feel a greater level of success in their overall scoring, but yet be just as well prepared to 
continue on to the next sections of the test. 

Other areas of the program that were cited as needing improvement are listed below: 

- the computer response time 

- the sound 

- the printing capabilities 

- the instructions to start and end the program 

- the instructions for the mouse 

In summary, the evaluation of Learning English Electronically revealed that the program is 
well liked by both students and teachers. Several recommendations are offered for potential 
improvements in the program based on student and teacher attitudes and suggestions, as well as 
observational data and performance test scores. LEE is a well regarded program and these 
recommendations simply offer the potential to further refine it and contribute to it's development. 
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TABLE 1 



SURVEY RESPONSES FROM TEACHERS 
SA = strongly agree 
A = agree 
N = no opinion 
D = disagree 
SD = strongly disagree 





SA 


A N 


D 


SD Mean 


1 . The modules are well organized. 




3 




2.00 


2. The skills taught in the modules 
are valuable for my students. 


2 


1 




1.33 


3. The modules are useful. 


1 


2 




1.67 


4. The stories aid in the learning process. 


2 


1 




1.33 


5. The rules are clearly stated. 




3 




2.00 


6. Instructions for the exercises are clear. 




3 




2.00 


7. There are enough practice exercises. 


2 


1 




1.33 


8. The modules are about the right length. 




3 




2.00 




7 


17 




1.71 



9. What are the major strengths of this program? 

Additional practice if the students take advantage of it. (1) 

Self-paced, nonthreatening. It has sound. (1) 

Grammar topics with content topics. (1) 

10. What are the major weaknesses of the program? 

The slowness. (1) 

No opinion. (1) 

1 1 . What could be done to improve the program? 

Make more students aware of it. (1) 

Improve its speed. (1) 

No opinion. (1) 

12. How do you use this program in your class? 

a. as a supplement b. as the main part of the class 

3 0 

13. Would you use this program again? 

Yes. (2) 

Always recommends extra services at Learning Center to all students. (1) 
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14. Other comments or suggestions? Needs to be completed. (1) 

TABLE 2 SURVEY RESPONSES FROM STUDENTS 








S A = strongly agree 
A = agree 
N = no opinion 
D = disagree 












SD = strongly disagree 










SD 


A 


N 


D 


SD 


Mean 


1 . The program was easy to use. 2 


6 






1 


2.11 


2. The sound with the story was helpful, 2 

I liked it. 


4 


1 






1.86 


3 . The practice exercises helped me learn. 5 


3 








1.38 


4. The instructions were easy to 2 

understand. 


5 


1 


1 




2.11 


5. Going back to the instructions helped 3 

me to learn. 


3 


1 






1.43 


6. The stories helped me to learn. 4 


4 








1.50 


7. 1 learned a lot from the program. 4 


4 








1.50 


8. The review helped me. 3 


3 


2 






1.88 


9. 1 liked the way the program told me 4 

the answers. 


4 








1.50 


10. 1 liked using the program. 6 


2 








1.25 


35 


38 


5~ 


T 


r 


1.65 



1 1 . The module was: 

a. too long 
1 



b. about right 
5 



12. Modules on the average were: 

a. too hard b. about right 

7 



c. too short 
1 



c. too easy 




7 
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13.1 used the modules because the teacher assigned them: 

a. yes b. no 

4 2 

14. 1 used the modules on my own: 

a. yes b.no 

4 3 

15. What did you like best about the program? 

The instruction helped me to learn. (1) 

Easy to understand. (1) 

Using the computer. (1) 

16. What did you like least about the program? 

Too slow. (3) 

No printing when desired. (1) 

17. What could be done to improve the program? Other comments or suggestions? 

More information on it throughout the college. (1) 

Make response time faster. (2) 

More exercises and more tests. (1) 
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TABLE 3 



MODULE-END PERFORMANCE TEST SCORES 



Section A 


Section B 


Section C 




100% 


50% 


100% 






70% 






70% 






50% 


50% 


100% 


Mean 73% 


75% 


75% 
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TABLE 4 



EVALUATOR OBSERVATIONS 



Evaluator observations are as follows: 

- first time users of the program need a lot of guidance from the 
instructors available in the lab 

- repeat users are able to manage with minimal teacher assistance 

- the program does work quite slowly and that seems to frustrate some of 
the students 

- the students don't rely on the written guide booklet very much 

- since the students can not print out the rules or all the exercises, they 
bring notebooks and take notes 

- students use the option of reviewing the rules in the program quite often 

- students who used the sound option in the program couldn't hear it very 
well and needed to use headphones 

- students who worked on the lessons about modals became frustrated 
because the computer only gave one answer as correct, when in reality 
more than one correct answer exists. Some proofreading and correcting 
also needs to be done as there are mistakes in some of the sentences. 
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